Recall Bias in Client-Reported Outcomes in Canine Orthopaedic Patients Using Clinical Metrology Instruments

Eirini Pappa; Thomas W. Maddox; Edward Crystal; Eithne J. Comerford; Andrew W. Tomlinson

doi:10.1055/s-0043-1771032

Veterinary and Comparative Orthopaedics and Traumatology, Table of Contents

Vet Comp Orthop Traumatol 2023; 36(06): 302-310
DOI: 10.1055/s-0043-1771032

Original Research

Recall Bias in Client-Reported Outcomes in Canine Orthopaedic Patients Using Clinical Metrology Instruments

Eirini Pappa

¹Department of Small Animal Clinical Science, Institute of Infection, Veterinary and Ecological Sciences, University of Liverpool, Neston, United Kingdom of Great Britain and Northern Ireland

,

Thomas W. Maddox

¹Department of Small Animal Clinical Science, Institute of Infection, Veterinary and Ecological Sciences, University of Liverpool, Neston, United Kingdom of Great Britain and Northern Ireland

²Department of Musculoskeletal and Aging Sciences, Institute of Life Course and Medical Sciences, Liverpool, United Kingdom of Great Britain and Northern Ireland

,

Edward Crystal

¹Department of Small Animal Clinical Science, Institute of Infection, Veterinary and Ecological Sciences, University of Liverpool, Neston, United Kingdom of Great Britain and Northern Ireland

,

Eithne J. Comerford

¹Department of Small Animal Clinical Science, Institute of Infection, Veterinary and Ecological Sciences, University of Liverpool, Neston, United Kingdom of Great Britain and Northern Ireland

²Department of Musculoskeletal and Aging Sciences, Institute of Life Course and Medical Sciences, Liverpool, United Kingdom of Great Britain and Northern Ireland

,

Andrew W. Tomlinson

¹Department of Small Animal Clinical Science, Institute of Infection, Veterinary and Ecological Sciences, University of Liverpool, Neston, United Kingdom of Great Britain and Northern Ireland

› Author Affiliations

Abstract

Objective The aim of this study was to determine the accuracy of client recollection of their dogs' preconsultation status using clinical metrology instruments such as the Liverpool Osteoarthritis in Dogs (LOAD) and Canine Brief Pain Inventory (CBPI) questionnaires in dogs presenting to a referral orthopaedic clinic.

Study Design This is a longitudinal prospective cohort study of client-owned dogs presenting for investigations of lameness (n = 217). LOAD and CBPI questionnaires were completed by the owners at the first consultation (T0). Owners were contacted at 2 (T1), 6 (T2), and 12 (T3) months and asked to recall their dogs' T0 status by completing another LOAD and CBPI questionnaire. The agreement between the T0 and recalled LOAD and CBPI scores was determined using the two-way mixed effects intraclass correlation coefficient (ICC). The Wilcoxon signed-rank test was used to determine the difference between scores.

Results For the LOAD scores, there was moderate agreement between T0 and T1 (ICC: 0.64) and T0 and T2 (ICC: 0.53) scores and poor agreement between T0 and T3 (ICC: 0.496). For the CBPI Pain Severity Scores, there was poor agreement between T0 and all three subsequent time points (ICC < 0.5). For the CBPI Pain Interference Scores, there was moderate agreement between T0 and T1 (ICC: 0.57) and T2 (ICC: 0.56) scores and poor agreement between T0 and T3 (ICC: 0.43).

Conclusion The LOAD and CBPI questionnaires are subject to recall bias. Studies reporting retrospectively acquired CMI data should be interpreted with caution.

Keywords

recall bias - LOAD - CBPI - clinical metrology instrument (CMI)

Full Text

References

References
1 Brown DC, Boston RC, Coyne JC, Farrar JT. Ability of the Canine Brief Pain Inventory to detect response to treatment in dogs with osteoarthritis. J Am Vet Med Assoc 2008; 233 (08) 1278-1283
2 Brown DC, Bell M, Rhodes L. Power of treatment success definitions when the Canine Brief Pain Inventory is used to evaluate carprofen treatment for the control of pain and inflammation in dogs with osteoarthritis. Am J Vet Res 2013; 74 (12) 1467-1473
3 Innes JF, Barr ARS. Can owners assess outcome following treatment of canine cruciate ligament deficiency?. J Small Anim Pract 1998; 39 (08) 373-378
4 Brown DC, Boston RC, Farrar JT. Comparison of force plate gait analysis and owner assessment of pain using the Canine Brief Pain Inventory in dogs with osteoarthritis. J Vet Intern Med 2013; 27 (01) 22-30
5 Quinn MM, Keuler NS, Lu Y, Faria MLE, Muir P, Markel MD. Evaluation of agreement between numerical rating scales, visual analogue scoring scales, and force plate gait analysis in dogs. Vet Surg 2007; 36 (04) 360-367
6 Waxman AS, Robinson DA, Evans RB, Hulse DA, Innes JF, Conzemius MG. Relationship between objective and subjective assessment of limb function in normal dogs with an experimentally induced lameness. Vet Surg 2008; 37 (03) 241-246
7 Oosterlinck M, Bosmans T, Gasthuys F. et al. Accuracy of pressure plate kinetic asymmetry indices and their correlation with visual gait assessment scores in lame and nonlame dogs. Am J Vet Res 2011; 72 (06) 820-825
8 Hercock CA, Pinchbeck G, Giejda A, Clegg PD, Innes JF. Validation of a client-based clinical metrology instrument for the evaluation of canine elbow osteoarthritis. J Small Anim Pract 2009; 50 (06) 266-271
9 Hudson JT, Slater MR, Taylor L, Scott HM, Kerwin SC. Assessing repeatability and validity of a visual analogue scale questionnaire for use in assessing pain and lameness in dogs. Am J Vet Res 2004; 65 (12) 1634-1643
10 Hielm-Björkman AK, Rita H, Tulamo RM. Psychometric testing of the Helsinki chronic pain index by completion of a questionnaire in Finnish by owners of dogs with chronic signs of pain caused by osteoarthritis. Am J Vet Res 2009; 70 (06) 727-734
11 Walton MB, Cowderoy E, Lascelles D, Innes JF. Evaluation of construct and criterion validity for the “Liverpool Osteoarthritis in Dogs” (LOAD) clinical metrology instrument and comparison to two other instruments. PLoS One 2013; 8 (03) e58125
12 Pinna S, Lambertini C, Grassato L, Romagnoli N. Evidence-based veterinary medicine: a tool for evaluating the healing process after surgical treatment for cranial cruciate ligament rupture in dogs. Front Vet Sci 2019; 6: 65
13 Essner A, Zetterberg L, Hellström K, Gustås P, Högberg H, Sjöström R. Psychometric evaluation of the Canine Brief Pain Inventory in a Swedish sample of dogs with pain related to osteoarthritis. Acta Vet Scand 2017; 59 (01) 44
14 Brown DC, Boston RC, Coyne JC, Farrar JT. Development and psychometric testing of an instrument designed to measure chronic pain in dogs with osteoarthritis. Am J Vet Res 2007; 68 (06) 631-637
15 Forster KE, Wills A, Torrington AM. et al. Complications and owner assessment of canine total hip replacement: a multicenter internet based survey. Vet Surg 2012; 41 (05) 545-550
16 De Sousa RJR, Parsons KJ, Owen MR. et al. Radiographic, surgeon and owner assessment of the Biomedtrix TATE elbow arthroplasty. Vet Surg 2016; 45 (06) 726-735
17 Swiderski JK, Palmer RH. Long-term outcome of distal femoral osteotomy for treatment of combined distal femoral varus and medial patellar luxation: 12 cases (1999-2004). J Am Vet Med Assoc 2007; 231 (07) 1070-1075
18 Terhaar HM, Muir P, Baker LA, Binversie EE, Chi J, Sample SJ. Contribution of habitual activity to cruciate ligament rupture in Labrador Retrievers. Vet Comp Orthop Traumatol 2020; 33 (02) 82-88
19 Henderson ER, Wills A, Torrington AM. et al. Evaluation of variables influencing success and complication rates in canine total hip replacement: results from the British Veterinary Orthopaedic Association Canine Hip Registry (collation of data: 2010-2012). Vet Rec 2017; 181 (01) 18
20 Godlonton S, Hernandez MA, Murphy M. Anchoring bias in recall data: evidence from central America. Am J Agric Econ 2018; 100 (02) 479-501
21 Krogsgaard MR, Brodersen J, Jensen J, Hansen CF, Comins JD. Potential problems in the use of patient reported outcome measures (PROMs) and reporting of PROM data in sports science. Scand J Med Sci Sports 2021; 31 (06) 1249-1258
22 Coughlin SS. Recall bias in epidemiologic studies. J Clin Epidemiol 1990; 43 (01) 87-91
23 Pellisé F, Vidal X, Hernández A, Cedraschi C, Bagó J, Villanueva C. Reliability of retrospective clinical data to evaluate the effectiveness of lumbar fusion in chronic low back pain. Spine 2005; 30 (03) 365-368
24 Murphy MT, Vardi R, Journeaux SF, Whitehouse SL. A patient's recollection of pre-operative status is not accurate one year after arthroplasty of the hip or knee. Bone Joint J 2015; 97-B (08) 1070-1075
25 Lingard EA, Wright EA, Sledge CB. Kinemax Outcomes Group. Pitfalls of using patient recall to derive preoperative status in outcome studies of total knee arthroplasty. J Bone Joint Surg Am 2001; 83 (08) 1149-1156
26 University of Liverpool. . Introducing the LOAD questionnaire: understanding, administering, and incorporating LOAD into clinical practice. Accessed June 20. 2023 at: https://www.pennvet.com/customer/wcm/connect/0caa373a-7925-4c0a-826f-1c34f6532376/LOAD_PrintableForm.pdf?MOD=AJPERES&CACHEID=ROOTWORKSPACE-0caa373a-7925-4c0a-826f-1c34f6532376-mN3Pgo
27 Rodrigues R, Silva PS, Cunha M, Vaz R, Pereira P. Can we assess the success of surgery for degenerative spinal diseases using patients'recall of their preoperative status?. World Neurosurg 2018; 115: e768-e773
28 Aleem IS, Currier BL, Yaszemski MJ. et al. Do cervical spine surgery patients recall their preoperative status? A cohort study of Recall Bias in Patient-reported Outcomes. Clin Spine Surg 2018; 31 (10) E481-E487
29 Aleem IS, Duncan J, Ahmed AM. et al. Do lumbar decompression and fusion patients recall their preoperative status? A cohort study of Recall Bias in Patient-reported Outcomes. Spine 2017; 42 (02) 128-134
30 Gotlin MJ, Kingery MT, Baron SL, McCafferty J, Jazrawi LM, Meislin RJ. Recall bias in retrospective assessment of preoperative patient-reported American shoulder and elbow surgeons scores in arthroscopic rotator cuff repair surgery. Am J Sports Med 2020; 48 (06) 1471-1475
31 Sumner-Smith G. Gait analysis and orthopaedic examination. In: Textbook of Small Animal Surgery. 2nd ed. Philadelphia, PA: W.B. Saunders; 1993. , p.1578
32 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33 (01) 159-174
33 Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15 (02) 155-163
34 Harmon-Jones E, Mills J. An introduction to cognitive dissonance theory and an overview of current perspectives on the theory. In: Harmon-Jones E. ed. Cognitive Dissonance: Progress on a Pivotal Theory in Social Psychology. Washington DC: American Psychological Association; 2019: 3-24
35 Schwartz CE, Bode R, Repucci N, Becker J, Sprangers MAG, Fayers PM. The clinical significance of adaptation to changing health: a meta-analysis of response shift. Qual Life Res 2006; 15 (09) 1533-1550
36 Schwartz CE, Sprangers MAG, Carey A, Reed G. Exploring response shift in longitudinal data. Pathol Health 2004; 19: 51-69
37 Blome C, Augustin M. Measuring change in quality of life: bias in prospective and retrospective evaluation. Value Health 2015; 18 (01) 110-115
38 Meyer T, Richter S, Raspe H. Agreement between pre-post measures of change and transition ratings as well as then-tests. BMC Med Res Methodol 2013; 13: 52
39 Sprangers MAG, Schwartz CE. Integrating response shift into health-related quality of life research: a theoretical model. Soc Sci Med 1999; 48 (11) 1507-1515
40 Sprangers MA, Van Dam FS, Broersen J. et al. Revealing response shift in longitudinal research on fatigue: the use of the thentest approach. Acta Oncol 1999; 38 (06) 709-718
41 Schwartz CE, Sprangers MAG. Guidelines for improving the stringency of response shift research using the thentest. Qual Life Res 2010; 19 (04) 455-464
42 Norman G. Hi! How are you? Response shift, implicit theories and differing epistemologies. Qual Life Res 2003; 12 (03) 239-249
43 Marsh J, Bryant D, MacDonald SJ. Older patients can accurately recall their preoperative health status six weeks following total hip arthroplasty. J Bone Joint Surg Am 2009; 91 (12) 2827-2837
44 Howell J, Xu M, Duncan CP, Masri BA, Garbuz DS. A comparison between patient recall and concurrent measurement of preoperative quality of life outcome in total hip arthroplasty. J Arthroplasty 2008; 23 (06) 843-849
45 Adogwa O, Elsamadicy AA, Cheng J, Bagley C. Assessing patient reported outcomes measures via phone interviews versus patient self-survey in the clinic: Are we measuring the same thing?. World Neurosurg 2016; 87: 230-234
46 Krosnick JA, Alwin DF. An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opin Q 1987; 51: 201-219
47 Mingay DJ, Greenwell MT. Memory bias and response-order effects. J Off Stat 1989; 5: 253-263
48 Sinclair M, O'Toole J, Malawaraarachchi M, Leder K. Comparison of response rates and cost-effectiveness for a community-based survey: postal, internet and telephone modes with generic or personalised recruitment approaches. BMC Med Res Methodol 2012; 12: 132
49 Pettitt RA, German AJ. Investigation and management of canine osteoarthritis. In Pract 2015; 37: 1-8
50 Johnston SA. Osteoarthritis. Joint anatomy, physiology, and pathobiology. Vet Clin North Am Small Anim Pract 1997; 27 (04) 699-723
51 Cook JL, Evans R, Conzemius MG. et al. Proposed definitions and criteria for reporting time frame, outcome, and complications for clinical orthopedic studies in veterinary medicine. Vet Surg 2010; 39 (08) 905-908
52 Compton J, Glass N, Fowler T. Evidence of selection bias and non-response bias in patient satisfaction surveys. Iowa Orthop J 2019; 39 (01) 195-201
53 Lie HC, Rueegg CS, Fosså SD, Loge JH, Ruud E, Kiserud CE. Limited evidence of non-response bias despite modest response rate in a nationwide survey of long-term cancer survivors-results from the NOR-CAYACS study. J Cancer Surviv 2019; 13 (03) 353-363
54 Gustavson K, Røysamb E, Borren I. Preventing bias from selective non-response in population-based survey studies: findings from a Monte Carlo simulation study. BMC Med Res Methodol 2019; 19 (01) 120

Supplementary Material

Supplementary Material